ANGELAKI Dora BAYES Multisensory Baylor  Nov 16 2012

from Baylor Colleg of Medicine, Houston, TX

this is in Clark Aud  at 4:30  SINTN series, Thurs

Why multisensory integration?

do we get benefit from multisen integration?look at a person's lips as he speaks

she shows  John Foxe  clip  Cereb Cortex 2006

much eazsier to understand... video shows woman saying GAME... but utterly cannot understand it
by listening only to audio...

Headingpersception...

optic flow... gives  direciton

(greg diangelis studies this)

Visual cues + optical flow...

VR system for monkeys

monkey on top of mtoin platform

heading discrim tsk...  2 altern forced choice task.. with monkey..

3 stim conditions

1)  vestibular condition... platform moves   but NO FLOE

2)  FLOW ONLY  (VR screen)

3) both together...

take home msg:

multisen itnegraiotn  reduces noise and incr  perception

sensory likelihood...

hypothesized value of stim:  uncertainty  = variance of distribution of  estimationof stimulus...

reliability is the inverse of variance...

Y axis = lielihood  and X axis is heading...

combining 2 sources of info

visual and vestibular...

use BAYES to combine...

if one cue is much less relaible it cannot help 
BUT if 2 cues have = relaibility then get  a  decreased variance of combo and incr reliability.

when  sig1 = sig1  then  sig of combo is 70% of  sig1  (ie decre variance  0.7 **2 = 50%

vary visual coherence  (of flow field)

this is from Gu  Nat Neurosci  11  2008...

******

is this reduciton in vriance also seen in single neuron response?

started from  MST  (studied b bill newsome)

responds to  both  stims....   platform motion  AND  optic flow...

stim 1:  move platform 
stim 2:  move optic flow..
or combo...

ROC analysiss  (see britten 1992)

one cell DID show combo effect  but
2 populations of cells:

some cells were congruent... incr firing rate  with both  vestib and visual

but some  were opposite (firing rate was opposite)

Congruent cells  do have  lower threshold for firing with both  congruent stim...

work done by Yong Gu in 2008 in Nat Neurosci...

did get a n improvement in threshold (lower by  sqrt of 2.

the more reliable cue will have the higher weights... the weights are prop to the inverse of the variance.

Optimal Cue Integration...

the McGurk Effect...

illustrates thsi... she shows  lips saying  GA GA... but sound alone sounds like  BA  BA...

seen alone the video seems to be saying DA DA...

Chris Fetsch in her lab...

place cues in conflict  eg  hae  vestib and visual off by 4 degrees.

take activity of entire population of neurons and combine...

they varied the reliability of cues from trial to trial...

behavior does correl with indiv cells...

multisensory field has been active for years

Super-additivity  known for years  from SC = super collic work

eg  Stein adn Stanford  Nat REv Neurosci 08...

huge incr in firing rate when combining 2 cues...

but  stein lab work only shows that super-additivity is only seen for threshold stims  (not like theirs)

Neuronal Combination Rule

wanted to find optimal weighting...

he small deviation from optimality can be traced down to level of firing of single cellss..

likely that  its some network property that cuases this...

DIVISIVE Normalization

firing of single neuron is normalized (divided by some norm of the population)

Ohshiro, Angelaki and DeAngelis...

******
Why multisensory integration?

1) cuz  sensory signals and their neural reps are noisy  but

2)  a single sensory cue can be ambiguous...

*****

One of the many snesory ambiguities in the visual system:  object motion  vs self-motion...
(can brain solve this with vision alone... eg walking ord riving)

use humans and monkeys...

everything so far was about the congruent cells...

hypothsis  abt congruent and opposite cells might play differ and complementary roles in solf-motion and object-motion

expt:  monkey is translated on plafrom

and  change  direction of flow and motion of objection...

look at tuning curves of indiv  MST cells...

takae home msg:

multisens integation if fund impt

Greg DeAngelis is at  U Rochester...

Q  where is convergence?
A it is a cortical property...
but ventral parietal area  (IP  area... is also a covnergence zone)
even if we inactiv  MST, animals can still do the task...

MT to MST